Inter-network and inter-protocol video conference privacy method, apparatus, and computer program product

ABSTRACT

A system, apparatus, method, and computer program product for detecting and correcting packet losses that adversely affect video quality. This packet loss feature may be employed as part of the secure video conference system, method and computer program product, or be employed in a non-secure system, method and computer program product.

CROSS REFERENCE TO RELATED APPLICATIONS

[0001] The present patent application is related and claims priority toprovisional U.S. application 60/423,393 filed on Nov. 4, 2002. Thepresent application is also related to co-pending U.S. applicationattorney docket number 243613US, filed on the same day as the presentapplication. The entire contents of both provisional U.S. application60/423,393 and co-pending U.S. application attorney docket number243610US are incorporated herein by reference.

BACKGROUND OF THE INVENTION

[0002] 1. Field of the Invention

[0003] This invention relates to systems, apparatuses, methods, andcomputer program products relating to establishing and maintaining highquality videoconferences between multiple nodes in which packet loss maybe an issue.

[0004] 2. Discussion of the Background

[0005] Video-conferencing is a ubiquitous form of the informationexchange in the modern era. A video-conference includes at least twostations exchanging video, audio, and other data to support of a virtualmeeting. The video is a stream of data made up of frames that includepictures and sound. Video signals are typically analog but may also bedigital. The digital information may or may not be compressed.

[0006] As recognized by the present inventors, a limitation withconventional systems is that by employing data compression in the videosignal, and subsequently transmitting the compressed data over acommunication link, compressed data may be lost or corrupted and causenoticeable losses in video quality. Moreover, a conventional techniquebegins by digitizing an input video signal to a format where it isrepresented by a number of frames per second, and a number of pixels perframe. The digitized video signal is then compressed to reduce therequired throughput demand on the communication link. The compressionmethod may be based on dividing each frame into blocks of pixels. Eachblock in a conventional system is compressed in one of the two followingways:

[0007] 1) As an INTRA block, i.e. the block is compressed independentlyof pixel values outside the current frame.

[0008] 2) As an INTER block, i.e. the block is compressed by forming theresidual data (difference) between the original pixel values and thecorresponding pixel values in a predicted block, and then compressingthe residual data. The predicted block is determined from one or morepreviously coded frames. The spatial displacement of a block between thecurrent frame and a previous frame is described by a motion vectorassociated with that block. The benefit of using INTER blocks is thesmall number bits required to represent the reduced dynamic range of thedifference signal. Thus, the compressed data for each INTER blockcontains two types of information: 1) motion vectors and 2) residualdata. (This is valid for many video coding standards including H.261,H.263, and H.264.)

[0009] To determine the predicted block, motion compensation is used.For each block in the original frame, a motion vector is used todescribe the spatial displacement between the original pixel values andthe corresponding pixel values in one of the previously coded frames.The predicted block is constructed from the displaced blocks in thepreviously coded frames. The motion vectors are determined for eachblock at the transmitter side by a motion estimation procedure, andtransmitted to the receiver side as side information.

[0010] The compressed data corresponding the each INTRA/INTER block arethen collected into transmission packets containing a certain number ofbytes. The data from one video frame are normally transmitted as severalpackets. At the receiver side, the INTRA blocks are decoded directly.The INTER blocks are decoded by first decoding the residual, and thenadding the predicted block using the corresponding motion vectors.

[0011] All communication links have some risk of generating transmissionerrors. In the video frame context, using an IP network for example, thetransmission errors will manifest themselves as “packet losses,” asillustrated in FIG. 1. As shown, a camera 711 produces video frames 701and applies the video frames 701 to a video encoder 713. The videoencoder formats the data in the frames into packets 702, which are thentransmitted over a network (such as IP network 715). In the network 715,packets are inadvertently lost or corrupted, such that received packetsmay contain a lost packet 703. A video decoder 717 is then faced withhaving to reproduce the video signal despite the fact the one or more ofthe packets have been lost. Consequently, the output of the videodecoder 717 contains some corrupted video frames 704, which appear asimperfect video images when produced on monitor 719. Becauseconventional methods do not typically employ the packet retransmissiontechniques, the decoder is faced with having to handle lost packets in away that minimizes the visual distortion for the end user.

[0012] In some situations, there can be two communication linksseparated by either an MCU or a Gateway as illustrated in FIG. 11 andFIG. 12. The main purpose of a Gateway is to reformat the compresseddata between an IP network and a non-IP network, typically an ISDNnetwork. The purpose of an MCU is to allow for several users toparticipate in the same conference. This implies that an MCU will haveseveral upstream encoders and downstream decoders connected. If theupstream encoder and downstream decoder pair is connected throughdifferent networks, the MCU performs reformatting in the same way as theGateway. However, if the picture resolution needs to be changed betweenthe upstream encoder and the downstream decoder, the MCU also needs toperform transcoding which includes decompression of the incoming datafollowed scaling of the decompressed video to the new resolution,followed by re-compression of the video with the new resolution. Inorder to perform transcoding, the MCU will then be equipped with aninternal encoder and decoder pair for each upstream/downstreamconnection.

[0013] If the encoder has information about a serious packet losssituation, it could modify its encoding format by adding more redundancyin the transmitted signal so as to avoid video distortion when the videoimages are reproduced at the destination terminal. As recognized by thepresent inventors, when there are no packet losses, INTER blocks aremuch more efficient than INTRA blocks since they normally use fewerbits. INTRA blocks are normally preferred only in case of scene changesand in areas with complex motion where a good approximation cannot befound in the previous frame. However, in the case of a packet lossduring transmission, INTER blocks are particularly vulnerable. This isbecause lost packets in an INTER block affects not only that particularframe, but also creates a propagation of packet errors in subsequentlydecoded frames. Thus, the error progressively contaminates multipleframes.

[0014] Assuming that the lost data cannot be retransmitted, the only wayto terminate the propagation of errors is to send an INTRA block,preferably as soon as possible after a packet loss has occurred. Asillustrated in FIG. 2, INTRA blocks 807 are interleaved at predeterminedintervals between series of INTER blocks 809. If a packet loss 805occurs in one of the INTER blocks 809, the all subsequently frames 803are corrupted until the next INTRA block is transmitted. Thus, asrecognized by the present inventors, there is a trade-off betweencompression efficiency using mostly INTER blocks, and robustness topacket loss using mostly INTRA blocks.

[0015] On method for intelligently managing the insertion of INTER andINTRA blocks is taught by Thomas Stockhammer and Thomas Wiegand in “H26LSimulation Results for Common Test Conditions for RTP/IP over3GPP/3GPP2”, VCEG-N38, ITU-Telecommunication Standardization Sector,STUDY GROUP 16 Question 6, Video Coding Experts Group (VCEG)”,Fourteenth Meeting: Santa Barbara, Calif., USA, 21-24 September, 2001,the entire contents of which are incorporated by reference. Among theproblems with this method, however, is the computational complexity ofthe encoder required to compensate for a sustained packet losssituation.

[0016] Ideally, the transmitter should get instant feedback from thereceiver and be able to react instantly by transmitting the INTRA blocksin the areas being affected by the loss. However, in many cases, thetransmitter receives limited, stale and/or inaccurate information aboutpacket losses detected at the receiver side. In particular, thetransmitter might receive only delayed information about packet losseswithout any reference to the particular frame, and to the particularblocks. Typically, the decoder sends a notification to the encoder eachtime a packet loss occurs. However, in order not to overload the system,a filter may be used to limit the frequency of these notifications to amaximum of M notifications per second.

[0017] What is required to address this limitation, as recognized by thepresent inventors, is an apparatus, system, method, and computer programproduct that based on the limited, delayed, and inaccurate notificationsdetects that there is a serious and sustained packet loss situation thatthe receiver is dealing with. Further, what is required, is an encoderthat, if this detection is made, can switch to a “robust mode”characterized by the intelligent insertion of INTRA blocks, instead ofINTER blocks, to achieve the best possible trade-off between efficientcompression and limited error propagation with minimal systemcomplexity.

[0018] In case of a packet loss, the task of the decoder is to concealthe lost data as well as it possibly can. If a packet loss occurs, oneor more blocks in that particular frame need to be replaced by somegenerated blocks that make the visible artifacts of the packet loss assmall as possible. One such process is to replace the missing blocks bythe corresponding blocks in the previous frame as described in TelenorResearch, “Definition of an error concealment model (TCON)”, LBC-95-186,ITI-TS SG15 Working Party 15/1, Expert's Group For Very LowBitrateVisual Telephony, Boston (20-23) Jun. 1995, the entire contents of whichare incorporated herein by reference. This process is illustrated inFIG. 3, where lines 3001 indicate block data from a previous framewithout motion vector information is used to estimate a lost block.Because this approach considers block data without consideration ofmotion vector information, this approach works reasonably well only whenthere is a high degree of similarity between consecutive frames (i.e.,no motion or almost no motion). However, this approach does not workwell when there is a low degree of similarity between consecutive frames(i.e., a large amount of motion.) More advanced conventional systems,therefore, have been developed to take into account motion vector info.

[0019] As shown in FIG. 4, concealment based on blocks from the sameposition in the previous frame can be further enhanced by combining themotion vector from the lost block with video (e.g., pixel) data from theprevious block. This is possible in situations where the block data islost but the motion vector data is not lost. Thus, concealment of a lostblock in block position (k,l) in frame n is performed using the lostblock's own motion vector, V_(n)(k,l).

[0020] However, typically a motion vector associated with a block islost with the rest of the data for that block. Thus, the challenge is tofind a good estimate V′_(n)(k,l) of the lost motion vector for the lostblock, particularly when there is a low degree of similarity betweenconsecutive frames. One method for estimating the motion vectors of lostblocks is taught by Stephan Wenger and Michael Horowitz in “ScatteredSlices: A New Error Resilience Tool for H.26L”, JVT-B027, Joint VideoTeam (JVT) of ISO/IEC MPEG and ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 andITU-T SG16 Q.6) 2nd Meeting, Geneva, CH, Jan. 29-Feb. 1, 2002, theentire contents of which are incorporated by reference. In this method,the motion vector of the lost block is estimated from motion vectors ofneighbor blocks within the same frame as the lost block with thefollowing equation:

V′ _(n)(k,l)=f(V _(n)(k−1,l−1),V _(n)(k−1,l),V _(n)(k−1,l+1),V_(n)(k,l−1),V _(n)(k,l+1),V _(n)(k+1,l−1),V _(n)(k+1,l),V _(n)(k+1,l+1))

[0021] where V_(n)(i,j) is the motion vector for block position (i,j) inframe n, and f( ) is some function. This situation is illustrated inFIG. 5. However, a problem with this method is that the method assumesthat neighbor blocks are not lost, which is not always true.

[0022] What is required, as discovered by the inventors, is an improvedmethod of estimating the contents of a lost block by finding an estimateof the motion vector for the lost block that does not rely upon theavailability of neighbor block motion vector information.

SUMMARY OF THE INVENTION

[0023] One object of the present invention is to provide an apparatus,system, method, and computer program product that conceals lost data atthe decoder side of a system that uses data compression over lines thatare vulnerable to data corruption. This is achieved by providing anaccurate estimate of motion vectors of blocks of video lost due topacket loss without relying on neighbor block information, particularlywhen there is a low degree of similarity between consecutive frames.

[0024] On the encoder side, the present invention also detects thatthere is some serious and sustained packet loss situation that thereceiver is dealing with. This detection is based on the limited,delayed, and inaccurate notifications about packet losses provided bythe receiver. If this detection is made, the encoder switches to a“robust mode” characterized by the intelligent insertion of INTRAblocks, instead of INTER blocks, to achieve an improved trade-offbetween efficient compression and limited error propagation. The Robustmode of the present invention contains a higher number of INTRA blockswhich spaced apart from each other based on an occurrence rate ofdetected packet losses. The higher the packet loss occurrence rate, thecloser the spacing of INTRA blocks. By spacing the INTRA blocks apartfrom one another in this fashion, the video disturbances caused bypacket losses are limited to just a few frames without unnecessarilydropping throughput rate or degrading video quality.

BRIEF DESCRIPTION OF THE DRAWINGS

[0025] A more complete appreciation of the present invention and many ofthe attendant advantages thereof will be readily obtained as the samebecomes better understood by reference to the following detaileddescriptions and accompanying drawings:

[0026]FIG. 1 is a signal flow diagram, illustrating the effect of packetloss on corrupted video frames output by a decoder;

[0027]FIG. 2 is an illustration of the progressive corruption of apacket error on sequential INTER frames;

[0028]FIG. 3 is a frame map showing a conventional block concealmentprocess that includes no motion vector information;

[0029]FIG. 4 is a frame map showing a conventional block concealmentprocess that combines motion vector information from a lost block withvideo data the corresponding block in a previous frame;

[0030]FIG. 5 is a frame map showing a conventional block concealmentprocess that uses motion vector data from a neighbor block in a currentframe;

[0031]FIG. 6 is a block diagram of a system that implements packet lossdetection, concealment, and management according to the presentinvention;

[0032]FIG. 7 is a frame map showing a distribution of INTRA and INTERblocks in normal mode processing and robust mode processing according tothe present invention;

[0033]FIG. 8 is a flowchart of a process for deciding whether normalmode or robust mode encoding should be performed according to thepresent invention;

[0034]FIG. 9 is a flowchart of a process showing how decoding withmotion vector estimation may be employed according to the presentinvention;

[0035]FIG. 10 is a frame map showing an embodiment of the invention'sblock concealment process that estimates a lost video data from bothblock and motion vector information from a previous frame;

[0036]FIG. 11 is a block diagram of a conventional IP-to-ISDN networkconnected by a conventional gateway device;

[0037]FIG. 12 is a block diagram of a IP-to-ISDN network connected by aconventional MCU;

[0038]FIG. 13 is a block diagram of a modified gateway or MCU accordingto an embodiment of the present invention;

[0039]FIG. 14 is a block diagram of a concealment device installed withor integral to the modified gateway or MCU of FIG. 12;

[0040]FIG. 15 is a flow diagram for concealment of corrupted packetscorresponding to the device of FIG. 13; and

[0041]FIG. 16 is a block diagram of a computer associated with thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0042] The following comments relate to the drawings, wherein likereference numerals designate identical or corresponding parts throughoutthe several views.

[0043]FIG. 6 shows the various components of the present invention. Theencoder 713, the IP network, 723, and the decoder 717 are present inboth conventional systems and systems that apply to the presentinvention. The sustained packet loss notification device 721 and theconcealment device 799 operate independently of each other, while thepacket loss detection device 719 is needed for both the sustained packetloss notification device 721 and the concealment device 799. This givesthe following three possibilities for implementation.

[0044] a) All components as illustrated in FIG. 6.

[0045] b) All components as illustrated in FIG. 6, except the Sustainedpacket loss detection device.

[0046] c) All components as illustrated in FIG. 6, except theConcealment device.

[0047]FIG. 6 is a block diagram of a system according to the presentinvention that employs a packet loss detection mechanism 719 to informan encoder 713 of a packet loss situation. The system includes anencoder 713 that sends video data over a communications network (wired,wireless or hybrid) to a decoder 717. Occasionally, the network (shownas an IP network 723) drops or corrupts some packets. Accordingly, apacket loss detection mechanism 719 monitors the video data for droppedor lost packets.

[0048] The packet loss detection mechanism 719 provides notificationmessages (at a periodic, scheduled, or per-lost-packet basis) to asustained packet loss detection mechanism 721 through either a dedicatedcommunication link, or a return link over the network. Thesenotifications typically do not need to identify exactly which packet waslost, however specific packet identification is possible in alternativeembodiments. The packet loss notifications are sent every time a packetloss occur, but usually not more frequent than M times per second.Typically, M is in the range of 1 to 10. The sustained packet lossdetection mechanism 721 determines if a sustained packet loss situationexists, and notifies the encoder 713 to switch modes if needed.

[0049] The sustained packet loss detection mechanism 721 detects asustained packet loss situation if there is sufficient periodicity inthe last n packet loss notifications. According to the invention,sufficient periodicity is determined as follows:

[0050] Let T₀, T₁, . . . and T_(N) be the time instances for the N mostrecent notifications.

[0051] Define the time differences as d₀=T₁−T₀, d₁=T₂−T₁, . . .d_(N−1)=T_(N)−T_(N−1).

[0052] Calculate the N×N ratios, r_(i,j)=|d_(i)|/|d_(j)| made up ofevery possible combination of two time differences.

[0053] If at least K of the N×N ratios are less than a predeterminedthreshold t, sufficient periodicity is detected.

[0054] Typical values for N, K, and t are 8, 32, and 0.9 respectivelyalthough other values may be used as well. For example, N may rangebetween 2 and 20 (typically, but not exclusively), K may range between 1through 400 (typically, although not exclusively) and t may rangebetween 0.5 through 0.99 (typically, but not exclusively). Moreover, itis not problematic to operate in a robust mode if there is sufficientthroughput capacity to handle the greater amount of data associated withINTRA blocks (assuming a lesser coding efficiency can be achieved for anINTRA block than for an INTRA block).

[0055] Whenever a sustained packet loss situation is detected, theencoder 713 switches to the robust mode under the direction of thesustained packet loss detection mechanism 721, which may be incorporatedinto the encoder 713. In an alternative embodiment not shown, thesustained packet loss detection mechanism 721 may be incorporated intothe decoder 717 rather than in the encoder 713.

[0056] The robust mode, as illustrated in FIG. 7, is characterized bythe insertion, by an encoder, of INTRA blocks instead of INTER blocksintelligently to achieve the best possible trade-off between efficientcompression and limited error propagation. As seen from FIG. 7, theRobust mode contains a higher number of INTRA blocks, spaced apart fromeach other based on an occurrence rate of detected packet losses. Thehigher the packet loss occurrence rate, the closer the spacing of INTRAblocks. By spacing the INTRA blocks apart from one another in thisfashion, the video disturbances caused by packet losses are limited tojust a few frames without unnecessarily dropping throughput rate ordegrading video quality.

[0057] According to the invention, robust mode contains two mechanisms,smart update and optimal mode selection. “Smart update” acts on packetloss notification by updating the entire picture using INTRA blocksduring a predetermined number of frames (e.g., 2 through 15). While astraightforward method sends the entire frame as INTRA immediately aftera packet loss notification, smart update uses several frames to updatethe picture. The benefit of this is to distribute the large number ofbits used for INTRA compared with INTER over several frames resulting ina more evenly distributed number of transmitted bits. During the smartupdate period, motion vectors are not allowed to point into areas of thepicture that has not yet been updated by an INTRA block.

[0058] “Optimal Mode Selection” is based on taking anticipated packetlosses into account when making a decision between INTRA and INTERblocks. In situations without packet loss, this mode selection isnormally done by analyzing the degree of similarity (such as by astatistical method like those discussed below, or a cross-correlationfunction to identify a “distance”) between the current frame and theprevious frame for that particular block. If there is a high degree ofsimilarity, INTER mode is chosen. For low degree of similarity(typically caused by scene changes or complex motion), INTRA mode ischosen. Without packet losses this is simply a compression efficiencyissue. Various statistical methods may be employed for making thelikeness test such as least squares fit, Chi-squared or other appliedstatistical methods, such as those described in Miller, R., “BeyondANOVA, Basics of Applied Statistics,” John Wiley & Sons, 1986, theentire contents of which being incorporated herein by reference.

[0059] When a sustained packet loss situation is detected, additionalcriteria for mode selection are used. These criteria include reliabilityof the reference block, and change in mean value. The reliability of ablock is used to modify the INTRA/INTER decision rule so that lowprobability favors an INTRA decision. The change in mean value from oneblock to another is used to modify the INTRA/INTER decision rule so thata large change in mean value favors INTRA decision. The decisioncriteria are described in more detail below.

[0060] In particular, the INTRA/INTER mode decision without sustainedpacket loss is made as follows:

[0061] 1) Calculate the sum of absolute differences (SAD) between theoriginal block pixels, c(i,j) and the pixels pointed to by the chosenmotion vector, p(i,j) as follows:${SAD}_{1} = {\sum\limits_{i,j}{{{c\left( {i,j} \right)} - {p\left( {i,j} \right)}}}}$

[0062] 2) Calculate the SAD between the original blocks of pixels andits mean value as follows:${SAD}_{2} = {{\sum\limits_{i,j}{{{c\left( {i,j} \right)} - M}}} + K}$

[0063]  where M is the mean value of the block, and where K is anempirically determined constant. A typical value for K is 500, but maybe in an inclusive range of 10 through 1000, for example.

[0064] 3) Make the mode selection as follows:

[0065] Choose INTRA mode if SAD₂<SAD₁

[0066] Choose INTER mode if SAD₂>SAD₁

[0067] However, in the sustained packet loss scenario, the reliabilityof the reference blocks depends on the period since it was last updatedas INTRA. In particular, the reliability, R_(n) of a reference block inframe no. n is determined as follows. For INTRA blocks: R_(n) = 1 − pFor INTER blocks with non-zero difference signal: R_(n) = (1 − p)R_(n−1)For INTER blocks with zero difference signals: R_(n) = R_(n−1)

[0068] Here, p is the packet loss probability and R_(n−1) is thereliability of the same block in the previous frame. In anotherimplementation, R_(n−1) can be replaced with an expression dependent onthe reliability of neighbor blocks to take the effects of non-zeromotion vectors into account.

[0069] The change in mean value, MD_(n) for a block in frame no. n iscalculated as follows:

MD _(n) =|M _(n) −M _(n−1)|

[0070] where M_(n) and M_(n−1) are the mean values of the block in frameno. n and n−1 respectively.

[0071] Thus, in the case of sustained packet loss, the INTRA/INTERdecision rule is modified by changing the value of SAD₁ as follows:${{SAD}_{1} = {\left\lbrack {\sum\limits_{i,j}{{{c\left( {i,j} \right)} - {p\left( {i,j} \right)}}}} \right\rbrack \left\lbrack {{\left( {{ar} + b} \right)\left( {{c\left( {1 - R_{n}} \right)} + {dpMD}_{n}} \right)} + 1} \right\rbrack}},$

[0072] where r is the bit rate of the compressed video stream measuredin kbps, p is the packet loss probability, and a, b, c, and d areempirically determined constants, depending on the communication system.Typical values for a, b, c and d, are 0.02, −6, 0.5, and (0.0001 through0.001) respectively. However, “a” may typically fall in an inclusiverange of 0.0 through 0.2; “b” may typically fall in an inclusive rangeof −100.0 through 100.0; “c” may typically fall in an inclusive range of0.0 through 100.0; and “d” may typically fall in an inclusive range of0.0001 through 0.01. The second term is a penalty term favoring theINTRA mode. The constants are chosen such that the penalty termincreases with the change in mean value MD_(n), the packet loss rate p,and the bit rate r, and decreases with the reliability R_(n). SAD₂ iscalculated in the manner previously described.

[0073]FIG. 8 is a flowchart for detecting packet loss and switching tothe robust mode in the encoder. The process begins in step S1300, wherethe packets losses are detected and collected, and notifications of thepacket losses are sent to the sustained packet loss detection mechanism721 (FIG. 6). Subsequently, the process proceeds to step S1303, wherepacket loss notifications are collected and analyzed for detecting a“sustained” packet loss situation. The process then proceeds to stepS1305 where an inquiry is made about whether a sustained packet losssituation exists. If the response to the inquiry is negative, theprocess proceeds to step S1307, where the normal INTER/INTRA decisionprocess is applied to select INTER or INTRA blocks, which are theninserted in the frame map in step S1311. However, if the response to theinquiry in step S1305 is affirmative, the process proceeds to step S1309where the modified INTER/INTRA decision process is invoked and a robustmode of operation is selected so as to offset the adverse affects ofsustained packet losses.

[0074] Also as shown in FIG. 6, the present invention may include aconcealment device 799. In this case, the packet loss detection device719 sends information to the concealment device about the identify ofwhich packets were received and which packets were lost. As opposed thethe notifications sent to the sustained packet loss detection device,the notifications sent to the concealment device typically identifiesexactly which packets were lost. According to the present invention, thecontents of lost blocks of video data may be estimated from previousand/or neighbor blocks of video data where the relationship betweenestimated motion vector and the motion vector of the corresponding blockfrom the previous frame is represented mathematically as follows:

V′ _(n)(k,l)=V _(n−1)(k,l).

[0075] That is, V′_(n)(k,l) is the estimated motion vector for frame nin block position (k,l) and V_(n−l)(k,l) is the actual motion vector offrame n−1 at block position (k,l). This is shown in FIG. 10.

[0076] Thus, in the present invention, the previous frame block motionvectors are used to generate a coarse estimate of the current frame fromthe previously decoded frame. In addition, an estimate of thecorresponding residuals is used to refine this estimate. In the presentinvention, the estimated residual is chosen as a predetermined value(either zero or another value).

[0077]FIG. 9 is a flowchart of a process flow according to the presentinvention where the decoder determines whether to operate withconcealment, or not. The process begins in step S1401 where there ispacket loss detection. The process then proceeds to step S1403, where aninquiry is made regarding whether packet losses were in fact detected.If so, the process proceeds to step S1405, where the motion estimationmechanism extracts motion vectors from one or more past frames, and/oradjacent blocks and provides them to the decoder in step S1407, wherethe decoder conceals the corrupted blocks. However, if the response tothe inquiry in step S1403 is negative, the process proceeds to stepS1409, where normal decoding is performed, and then the processcontinues in step S1411.

[0078] In other embodiments of the present invention, the previouslydescribed techniques for handling packet loss can be extended and/oradapted to the case where a gateway or an MCU is used in the packet lossscenario of FIG. 1. In one of these embodiments, the gateway 1505 issituated between an IP network 1503, which is fed data by an encoder1501, and an ISDN network 1507, which feeds data to a decoder 1509 asillustrated in FIG. 11. In another of these embodiments, an MCU 1605 canbe situated between an IP network 1603, which is fed data by an encoder1601 and another network 1607 (e.g., an ISDN network or a second IPnetwork), which feeds data to a decoder 1609, as illustrated in FIG. 12.It should be clear to one skilled in the art that the another network1607 may also be an ATM network, an MPLS network, or any othercontinuous stream network.

[0079] In the IP network 1503, the compressed video data from theencoder is divided into IP packets, each packet containing a certainnumber of bytes. Since each packet contains a header with a uniqueidentifier, lost packets can easily be detected at the receiving end.

[0080] When the another network 1607 is an ISDN network, the compressedvideo data is a continuous stream of bytes. Since there is no packetheader in an ISDN network, detection of lost data is more difficult. Onthe other hand, loss of data occurs extremely rare in an ISDN network.

[0081] One function of the gateway 1505 is to reformat compressed datafrom a packetized byte stream to a continuous byte stream. One problemwith conventional systems which are subject to noise or other signalloss, packet losses in the IP network will propagate to the ISDN networkas a missing sub-sequence of the continuous byte stream. Since thedownstream decoder 1509 has no knowledge about the initial packet loss,and since errors in a continuous byte stream are hard to detect, thesystem of FIG. 6 will fail since:

[0082] 1) The encoder 1501 is not notified about the packet loss andfails to take appropriate action.

[0083] 2) The decoder 1509 does not initiate concealment.

[0084] One embodiment of the present invention solves this problem byadapting the previously described packet loss detection and concealmentmethods as shown in FIG. 13. Here, video encoder 1701 transmitscompressed video data packets via IP network 1703 to a gateway or MCU1705. Gateway or MCU 1705 relays the packets to video decoder 1709 ifthe connecting network 1707 is an IP network. Gateway or MCU 1705reformats the packets to a compatible format before relaying the data tovideo decoder 1709 if the connecting network 1707 is not an IP network(e.g., an ISDN network).

[0085] In either case, in one embodiment, gateway or MCU 1705 includes aconcealment device 1713 and a packet loss detection device 1711 whilevideo encoder 1701 also includes a sustained packet loss detectionmechanism (not shown). In a second embodiment (not shown), gateway orMCU 1705 includes a packet loss detection device 1711 but does notinclude a concealment device 1713 while video encoder 1701 does includea sustained packet loss detection mechanism. In a third embodiment (notshown), gateway or MCU 1705 includes a concealment device 1713 and apacket loss detection device 1711 while video encoder 1701 does notinclude a sustained packet loss detection mechanism.

[0086] In the embodiment of FIG. 13, encoder 1701 relays video data toIP network 1703. Gateway or MCU 1705 further relays the video data tovideo decoder 1709 via ISDN network 1707. The gateway or MCU 1705includes, or is co-located with, a packet loss detection device 1711 anda concealment device 1713.

[0087] In this embodiment, packet loss detection by packet lossdetection device 1711 can be done in the same way as previouslydescribed relative to FIG. 6, to include informing encoder 1701 of thepresence of lost packets. Also, the upstream sustained packet lossdetection device (not shown) performs in the same way as previouslydescribed.

[0088] Also, for an MCU 1705 transcoding by compression anddecompression, concealment device 1713 estimates the lost video blocks,inserts the estimated data within the uncorrupted data, before the MCUcompresses the resulting video data into the downstream network format(e.g., ISDN) without degrading video-conference performance. In thisscenario, the previously described method of estimating a missing videoframe based on previous frame motion vectors and residuals is directlyapplicable.

[0089] However, for a gateway or an MCU 1705 relaying packets withoutdoing transcoding, concealment device 1713 must first decompress theincoming packets, estimate the lost video frames, compress the estimateddata, and insert the compressed estimated data into the compressedstream or packets without causing undue delay. This embodiment operatesdifferently than was previously described. In this scenario, thepreviously described method of estimating a missing video frame based onprevious frame motion vectors and residuals is augmented with anadditional step of re-compressing the estimated video data and insertingthe re-compressed video data into the downstream packet stream.

[0090] Therefore, as shown in FIG. 14, for a gateway or an MCU 1705relaying packets without doing transcoding, a sequence of packets 1801is sent by encoder 1701. Sequence 1801 becomes corrupted in IP network1703 resulting in a sequence 1803 with a missing packet. Concealmentdevice 1713, which includes a decoder and encoder, replaces the lostpacket with an estimated packet 1805 so that gateway or MCU 1705 sends asequence 1807 that includes estimated packet 1805.

[0091] The process of decompressing, estimating, and recompressing isshown in FIG. 15 as follows:

[0092] Step S1801—For each packet received, decompress the packet anddecode the motion vectors from the compressed data;

[0093] Step S1803—For the lost packet, estimate the corresponding motionvectors, using the same method as for concealment described relative toFIG. 9;

[0094] Step S1805—Compress the estimated motion vectors;

[0095] Step S1807—Reset the corresponding residuals to a predeterminedvalue (either zero or another value);

[0096] Step S1809—Compress the reset residuals;

[0097] Step S1811—Combine the compressed estimated motion vectors andcompressed reset residuals to form an estimated packet; and

[0098] Step S1813—Output the estimated packet for inclusion in thepacket stream output by gateway or MCU 1705.

[0099] Apart from performing reliable loss detection and concealment,performing packet loss detection and concealment in the gateway or MCUalso has the benefit of minimizing the round-trip delay associated withpacket loss notifications and the corresponding encoder reply.

[0100]FIG. 16 is a block diagram of a computer system 2001 upon which anembodiment of the present invention may be implemented. It should benoted however, that the present system need not be based on a personalcomputer (PC) configuration, but rather a custom processor-based system(such as a software and/or hardware modified Tandberg 6000, or TandbergMCU) that does not include the features of a general purpose computermay be used as well. Nevertheless, because the actual hardwareconfiguration used to support the present invention, is not sorestricted, an example of PC-based system is now provided. The computersystem 2001 includes a bus 2002 or other communication mechanism forcommunicating information, and a processor 2003 coupled with the bus2002 for processing the information. The computer system 2001 alsoincludes a main memory 2004, such as a random access memory (RAM) orother dynamic storage device (e.g., dynamic RAM (DRAM), static RAM(SRAM), and synchronous DRAM (SDRAM)), coupled to the bus 2002 forstoring information and instructions to be executed by processor 2003.In addition, the main memory 2004 may be used for storing temporaryvariables or other intermediate information during the execution ofinstructions by the processor 2003. The computer system 2001 furtherincludes a read only memory (ROM) 2005 or other static storage device(e.g., programmable ROM (PROM), erasable PROM (EPROM), and electricallyerasable PROM (EEPROM)) coupled to the bus 2002 for storing staticinformation and instructions for the processor 2003.

[0101] The computer system 2001 also includes a disk controller 2006coupled to the bus 2002 to control one or more storage devices forstoring information and instructions, such as a magnetic hard disk 2007,and a removable media drive 2008 (e.g., floppy disk drive, read-onlycompact disc drive, read/write compact disc drive, compact disc jukebox,tape drive, and removable magneto-optical drive). The storage devicesmay be added to the computer system 2001 using an appropriate deviceinterface (e.g., small computer system interface (SCSI), integrateddevice electronics (IDE), enhanced-IDE (E-IDE), direct memory access(DMA), or ultra-DMA).

[0102] The computer system 2001 may also include special purpose logicdevices (e.g., application specific integrated circuits (ASICs)) orconfigurable logic devices (e.g., simple programmable logic devices(SPLDs), complex programmable logic devices (CPLDs), and fieldprogrammable gate arrays (FPGAs)).

[0103] The computer system 2001 may also include a display controller2009 coupled to the bus 2002 to control a display 2010, such as acathode ray tube (CRT), for displaying information to a computer user.The computer system includes input devices, such as a keyboard 2011 anda pointing device 2012, for interacting with a computer user andproviding information to the processor 2003. The pointing device 2012,for example, may be a mouse, a trackball, or a pointing stick forcommunicating direction information and command selections to theprocessor 2003 and for controlling cursor movement on the display 2010.In addition, a printer may provide printed listings of data storedand/or generated by the computer system 2001.

[0104] The computer system 2001 performs a portion or all of theprocessing steps of the invention in response to the processor 2003executing one or more sequences of one or more instructions contained ina memory, such as the main memory 2004. Such instructions may be readinto the main memory 2004 from another computer readable medium, such asa hard disk 2007 or a removable media drive 2008. One or more processorsin a multi-processing arrangement may also be employed to execute thesequences of instructions contained in main memory 2004. In alternativeembodiments, hard-wired circuitry may be used in place of or incombination with software instructions. Thus, embodiments are notlimited to any specific combination of hardware circuitry and software.

[0105] As stated above, the computer system 2001 includes at least onecomputer readable medium or memory for holding instructions programmedaccording to the teachings of the invention and for containing datastructures, tables, records, or other data described herein. Examples ofcomputer readable media are compact discs, hard disks, floppy disks,tape, magneto-optical disks, PROMs (EPROM, EEPROM, flash EPROM), DRAM,SRAM, SDRAM, or any other magnetic medium, compact discs (e.g., CD-ROM),or any other optical medium, punch cards, paper tape, or other physicalmedium with patterns of holes, a carrier wave (described below), or anyother medium from which a computer can read.

[0106] Stored on any one or on a combination of computer readable media,the present invention includes software for controlling the computersystem 2001, for driving a device or devices for implementing theinvention, and for enabling the computer system 2001 to interact with ahuman user (e.g., print production personnel). Such software mayinclude, but is not limited to, device drivers, operating systems,development tools, and applications software. Such computer readablemedia further includes the computer program product of the presentinvention for performing all or a portion (if processing is distributed)of the processing performed in implementing the invention.

[0107] The computer code devices of the present invention may be anyinterpretable or executable code mechanism, including but not limited toscripts, interpretable programs, dynamic link libraries (DLLs), Javaclasses, and complete executable programs. Moreover, parts of theprocessing of the present invention may be distributed for betterperformance, reliability, and/or cost.

[0108] The term “computer readable medium” as used herein refers to anymedium that participates in providing instructions to the processor 2003for execution. A computer readable medium may take many forms, includingbut not limited to, non-volatile media, volatile media, and transmissionmedia. Non-volatile media includes, for example, optical, magneticdisks, and magneto-optical disks, such as the hard disk 2007 or theremovable media drive 2008. Volatile media includes dynamic memory, suchas the main memory 2004. Transmission media includes coaxial cables,copper wire and fiber optics, including the wires that make up the bus2002. Transmission media also may also take the form of acoustic orlight waves, such as those generated during radio wave and infrared datacommunications.

[0109] Various forms of computer readable media may be involved incarrying out one or more sequences of one or more instructions toprocessor 2003 for execution. For example, the instructions mayinitially be carried on a magnetic disk of a remote computer. The remotecomputer can load the instructions for implementing all or a portion ofthe present invention remotely into a dynamic memory and send theinstructions over a telephone line using a modem. A modem local to thecomputer system 2001 may receive the data on the telephone line and usean infrared transmitter to convert the data to an infrared signal. Aninfrared detector coupled to the bus 2002 can receive the data carriedin the infrared signal and place the data on the bus 2002. The bus 2002carries the data to the main memory 2004, from which the processor 2003retrieves and executes the instructions. The instructions received bythe main memory 2004 may optionally be stored on storage device 2007 or2008 either before or after execution by processor 2003.

[0110] The computer system 2001 also includes a communication interface2013 coupled to the bus 2002. The communication interface 2013 providesa two-way data communication coupling to a network link 2014 that isconnected to, for example, a local area network (LAN) 2015, or toanother communications network 2016 such as the Internet. For example,the communication interface 2013 may be a network interface card toattach to any packet switched LAN. As another example, the communicationinterface 2013 may be an asymmetrical digital subscriber line (ADSL)card, an integrated services digital network (ISDN) card or a modem toprovide a data communication connection to a corresponding type ofcommunications line. Wireless links may also be implemented. In any suchimplementation, the communication interface 2013 sends and receiveselectrical, electromagnetic or optical signals that carry digital datastreams representing various types of information.

[0111] The network link 2014 typically provides data communicationthrough one or more networks to other data devices. For example, thenetwork link 2014 may provide a connection to another computer through alocal network 2015 (e.g., a LAN) or through equipment operated by aservice provider, which provides communication services through acommunications network 2016. The local network 2014 and thecommunications network 2016 use, for example, electrical,electromagnetic, or optical signals that carry digital data streams, andthe associated physical layer (e.g., CAT 5 cable, coaxial cable, opticalfiber, etc). The signals through the various networks and the signals onthe network link 2014 and through the communication interface 2013,which carry the digital data to and from the computer system 2001 maybeimplemented in baseband signals, or carrier wave based signals. Thebaseband signals convey the digital data as unmodulated electricalpulses that are descriptive of a stream of digital data bits, where theterm “bits” is to be construed broadly to mean symbol, where each symbolconveys at least one or more information bits. The digital data may alsobe used to modulate a carrier wave, such as with amplitude, phase and/orfrequency shift keyed signals that are propagated over a conductivemedia, or transmitted as electromagnetic waves through a propagationmedium. Thus, the digital data may be sent as unmodulated baseband datathrough a “wired” communication channel and/or sent within apredetermined frequency band, different than baseband, by modulating acarrier wave. The computer system 2001 can transmit and receive data,including program code, through the network(s) 2015 and 2016, thenetwork link 2014, and the communication interface 2013. Moreover, thenetwork link 2014 may provide a connection through a LAN 2015 to amobile device 2017 such as a personal digital assistant (PDA) laptopcomputer, or cellular telephone.

[0112] Obviously, numerous modifications and variations of the presentinvention are possible in light of the above teachings. It is thereforeto be understood that within the scope of the appended claims, theinvention may be practiced otherwise than as specifically describedherein.

1. A video-conferencing system, comprising: a decoder; an encoderconnected to the decoder by a network; a packet loss detection mechanismco-located with the decoder, the packet loss detection mechanismconfigured to detect a lost packet and to transmit one of a packet lossnotification message and a lost packet identification message; and oneof a sustained packet loss detection mechanism co-located with one ofthe encoder and the packet loss detection mechanism, the sustainedpacket loss detection mechanism configured to receive the packet lossnotification message and to inform the encoder that the packet lossnotification message indicates a presence of one of a first and a secondpredetermined packet loss scenario, and an error concealment deviceco-located with the decoder and connected to the packet loss mechanism,the error concealment device configured to replace motion vector datalost from a current frame with motion vector data from a previous framein response to the lost packet identification message.
 2. Thevideo-conferencing system of claim 1, wherein the encoder is configuredto insert an INTER or INTRA block according to a first and a secondinsertion method if the packet loss notification message indicates thepresence of the first and the second predetermined packet lossscenarios, respectively.
 3. The video-conferencing system of claim 1,wherein the first predetermined packet loss scenario comprises: aperiodicity in a last N packet loss notifications that exceeds apredetermined packet loss periodicity value.
 4. The video-conferencingsystem of claim 3, wherein the first predetermined packet loss scenariocomprises: at least K of N×N ratios which are less than a predeterminedthreshold t, where T₀, T₁, . . . and T_(N) are a set of time instancescorresponding to a set of N most recent packet loss notifications; d_(i)is a plurality of time differences d₀=T₁−T₀, d₁=T₂−T₁, . . .d_(N−1)=T_(N)−T_(N−1) between packet loss notifications; and the N×Nratios equal r_(i,j)=|d_(i)|/|d_(j)| corresponding to every possiblecombination of two time differences.
 5. The video-conferencing system ofclaim 1, wherein the encoder is configured to insert a plurality ofINTRA blocks separated by a predetermined distance corresponding to apacket loss occurrence rate.
 6. The video-conferencing system of claim5, wherein the predetermined distance is configured to vary such that ahigher packet loss occurrence rate results in a closer spacing ofinserted INTRA blocks.
 7. The video-conferencing system of claim 5,wherein the encoder is configured to update an entire picture usingINTRA blocks during a predetermined number of frames.
 8. Thevideo-conferencing system of claim 7, wherein the encoder is configuredto prevent a motion vector from pointing into an area of the entirepicture that has not yet been updated by an INTRA block.
 9. Thevideo-conferencing system of claim 1, wherein the encoder is configuredto take anticipated packet losses into account when making a decisionbetween inserting an INTRA block and inserting an INTER block.
 10. Thevideo-conferencing system of claim 1, wherein the encoder is furtherconfigured to determine one of a reliability of a reference block and achange in a reference block mean value.
 11. The video-conferencingsystem of claim 10, wherein the encoder is configured to determine areliability of a reference block on the basis of a period since thereference block was last updated as INTRA.
 12. The video-conferencingsystem of claim 11, wherein the encoder is configured to determine areliability R_(n) of a reference block in a frame n is determined asfollows: R_(n)=1−p for INTRA blocks; R_(n)=(1−p)R_(n−1) for INTER blockswith non-zero difference signal; and R_(n)=R_(n−1) for INTER blocks withzero difference signals, where p is a packet loss probability andR_(n−1) is one of a reliability of a same block in a previous frame andan expression dependent on a reliability of neighbor blocks thatincludes non-zero motion vectors effects.
 13. The video-conferencingsystem of claim 12, wherein the encoder is configured to determine achange in mean value, MD_(n) for a block in a frame n as follows:MD_(n)=|M_(n)−M_(n−1) where M_(n) and M_(n−1) are mean values of a blockin frame n and n−1, respectively.
 14. The video-conferencing system ofclaim 13, wherein the encoder is configured to determine a sum ofabsolute differences between original block pixels, c(i,j) and pixelspointed to by a chosen motion vector, p(i,j) as follows${{SAD}_{1} = {\left\lbrack {\sum\limits_{i,j}{{{c\left( {i,j} \right)} - {p\left( {i,j} \right)}}}} \right\rbrack \left\lbrack {{\left( {{ar} + b} \right)\left( {{c\left( {1 - R_{n}} \right)} + {dpMD}_{n}} \right)} + 1} \right\rbrack}},$

where r is a bit rate of a compressed video stream measured in kbps, pis a packet loss probability, and a, b, c, and d are empiricallydetermined constants, corresponding to a communication system, determinea sum of absolute differences between the original blocks of pixels anda mean value as follows${{SAD}_{2} = {{\sum\limits_{i,j}{{{c\left( {i,j} \right)} - M_{n}}}} + K}},$

where K is an empirically determined constant; and insert the INTRAblock if SAD₂<SAD₁ or insert the INTER block if SAD₂≧SAD₁.
 15. Thevideo-conferencing system of claim 14, wherein the encoder is configuredto choose constants a, b, c, and d such that a penalty term increaseswith a change in mean value MD_(n), the packet loss rate p, and the bitrate r, and decreases with reliability R_(n).
 16. The video-conferencingsystem of claim 1, wherein the error concealment device is configured toestimate a motion vector V′_(n)(k, 1) of a lost frame n in blockposition (k,l) with an actual motion vector V_(n−1)(k,l) of a previousframe n−1 at block position (k,l).
 17. The video-conferencing system ofclaim 16, wherein the error concealment device is configured to set aresidual of the lost frame n to a predetermined residual value.
 18. Thevideo-conferencing system of claim 17, wherein the error concealmentdevice is configured to set the residual of the lost frame n to zero.19. A method for video-conferencing, comprising: collecting one ofpacket loss notifications and a lost packet identifier; and one ofdetermining if the collected packet loss notifications correspond to oneof a first and a second predetermined packet loss scenario, andreplacing video data lost in a current frame using motion vectorinformation from a previous frame in response to the lost packetidentifier.
 20. The video-conferencing method of claim 19, furthercomprising: inserting one of an INTRA and an INTER block according to afirst and a second method if the collected packet loss notificationscorrespond to the first and the second predetermined packet lossscenario, respectively.
 21. The video-conferencing method of claim 19,further comprising: detecting the first packet loss scenario if aperiodicity in last n packet loss notifications exceeds a predeterminedpacket loss periodicity value.
 22. The video-conferencing method ofclaim 21, further comprising: detecting the first packet loss scenarioif at least K of N×N ratios are less than a predetermined threshold t,where T₀, T₁, . . . and T_(N) are a set of time instances correspondingto a set of N most recent notifications; d_(i) is a plurality of timedifferences d₀=T₁−T₀, d₁=T₂−T₁, . . . d_(N−1)=T_(N)−T_(N−1); and the N×Nratios equal r_(i,j)=|d_(i)|/|d_(j)| corresponding to every possiblecombination of two time differences.
 23. The video-conferencing methodof claim 19, further comprising: inserting a plurality of INTRA blocksseparated by a predetermined distance upon receipt of the packet lossnotification, wherein the predetermined distance corresponds to a packetloss occurrence rate.
 24. The video-conferencing method of claim 19,further comprising: varying the predetermined distance such that ahigher packet loss occurrence rate results in a closer spacing ofinserted INTRA blocks.
 25. The video-conferencing method of claim 19,further comprising: updating an entire picture using INTRA blocks duringa predetermined number of frames.
 26. The video-conferencing method ofclaim 19, further comprising: preventing a motion vector from pointinginto areas of the entire picture that has not yet been updated by anINTRA block.
 27. The video-conferencing method of claim 19, furthercomprising: estimating anticipated packet losses when making a decisionbetween inserting an INTRA and inserting an INTER blocks.
 28. Thevideo-conferencing method of claim 19, further comprising: determiningone of a reliability of a reference block and a change in mean valuewhen a packet loss scenario is detected.
 29. The video-conferencingmethod of claim 28, further comprising: determining a reliability R_(n)of a reference block in a frame n on the basis of a period since thereference block was last updated as INTRA.
 30. The video-conferencingmethod of claim 29, wherein the step of determining a reliability R_(n)comprises: determining R_(n)=1−p for INTRA blocks; R_(n)=(1−p)R_(n−1)for INTER blocks with non-zero difference signal; and R_(n)=R_(n−1) forINTER blocks with zero difference signals, where p is a packet lossprobability and R_(n−1) is one of a reliability of a same block in aprevious frame and an expression dependent on a reliability of neighborblocks that includes non-zero motion vectors effects.
 31. Thevideo-conferencing method of claim 30, further comprising: determining achange in mean value, MD_(n) for a block in a frame n as follows: MD_(n) =|M _(n) −M _(n−1)| where M_(n) and M_(n−1) are mean values of ablock in frames n and n-1, respectively.
 32. The video-conferencingmethod of claim 31, further comprising: determining, in a presence ofthe predetermined packet loss scenario, SAD₁ as follows:${{SAD}_{1} = {\left\lbrack {\sum\limits_{i,j}{{{c\left( {i,j} \right)} - {p\left( {i,j} \right)}}}} \right\rbrack \left\lbrack {{\left( {{ar} + b} \right)\left( {{c\left( {1 - R_{n}} \right)} + {dpMD}_{n}} \right)} + 1} \right\rbrack}},$

where r is a bit rate of a compressed video stream measured in kbps, pis a packet loss probability, and a, b, c, and d are empiricallydetermined constants, corresponding to a communication method;determining a sum of absolute differences between the original blocks ofpixels and a mean value as follows: determine a sum of absolutedifferences between the original blocks of pixels and a mean value asfollows${{SAD}_{2} = {{\sum\limits_{i,j}{{{c\left( {i,j} \right)} - M_{n}}}} + K}},$

where K is an empirically determined constant; and inserting the INTRAblock if SAD₂<SAD₁ or inserting the INTER block if SAD₂≧SAD₁.
 33. Thevideo-conferencing method of claim 32, further comprising: choosingconstants a, b, c, and d such that a penalty term increases with achange in mean value MD_(n), the packet loss rate p, and the bit rate r,and decreases with reliability R_(n).
 34. The video-conferencing methodof claim 19, wherein the step of replacing video data lost comprises:setting an estimated motion vector V′_(n)(k,l) for frame n in blockposition (k,l) equal to an actual motion vector V_(n−1)(k,l) of aprevious frame n−1 at block position (k,l).
 35. The video-conferencingmethod of claim 34, wherein the step of replacing video data lostfurther comprises: setting a residual corresponding to the actual motionvector to a predetermined residual value.
 36. The video-conferencingmethod of claim 35, wherein the step of setting a residual comprises:setting the residual to zero.
 37. A video-conferencing interface deviceconnecting two networks, comprising: a decoder connected to an upstreamencoder by a first of the two networks; an encoder connecting thedecoder to a downstream decoder by a second of the two networks; apacket loss detection mechanism connected to the decoder configured todetect a lost packet and to transmit one of a packet loss notificationmessage and a lost packet identifier; and one of a sustained packet lossdetection mechanism connected to the packet loss detection mechanism andthe upstream encoder, the sustained packet loss detection mechanismconfigured to receive the packet loss notification message and to informthe upstream encoder that the packet loss notification message indicatesa presence of one of a first and a second predetermined upstream packetloss scenario, and an error concealment device connected to the decoderand the packet loss detection mechanism, the error concealment deviceconfigured to replace a lost motion vector with a replacement motionvector from a previous frame.
 38. The video-conferencing gateway deviceof claim 37, wherein the encoder is connected to a second sustainedpacket loss notification device configured to identify a presence of afirst and a second predetermined downstream packet loss scenario, theencoder further configured to insert an INTER or INTRA block accordingto one of a first and a second method corresponding to the presence ofthe first and the second predetermined downstream packet loss scenario.39. A video-conferencing interface device connecting two networks,comprising: a packet loss detection mechanism configured to detect alost packet and to transmit a lost packet identifier; and an errorconcealment device connected to the packet loss detection mechanism, theerror concealment device including a decoder and an encoder, the errorconcealment device configured to replace a lost motion vector with areplacement motion vector from a previous frame.
 40. Thevideo-conferencing interface device of claim 39, wherein the errorconcealment device is further configured to decompress incoming packetsof compressed video data, detect a lost packet of decompressed videodata, estimate a lost packet motion vector from a corresponding previousframe motion vector, compress the estimated lost packet motion vector,reset a residual corresponding to the lost packet motion vector to apredetermined value, compress the reset residual, combine the compressedestimated motion vector and compressed reset residual to form anestimated packet, and output the estimated packet to the second network.41. The video-conferencing interface device of claim 40, wherein: thepacket loss detection device is further configured to transmit a packetloss notification message.
 42. The video-conferencing interface deviceof claim 41, further comprising: a sustained packet loss detectionmechanism connected to the packet loss detection mechanism and to anupstream encoder, the sustained packet loss detection mechanismconfigured to receive the packet loss notification message from thepacket loss detection device and to inform the upstream encoder that thepacket loss notification message indicates a presence of one of a firstand a second predetermined upstream packet loss scenario.
 43. Avideo-conferencing encoder, comprising: a connection to a downstreamdecoder; wherein the encoder is configured to insert an INTER or INTRAblock according to one of a first and a second method, said first andsecond methods selected in response to a presence of a first and asecond predetermined downstream decoder packet loss scenario,respectively.
 44. The video-conferencing encoder of claim 43, furthercomprising: a sustained packet loss detection mechanism configured toreceive a downstream packet loss notification message from a packet lossdetection mechanism co-located with the downstream decoder, thedownstream packet loss notification message configured to indicate apresence of one of the first and the second predetermined downstreampacket loss scenario.
 45. A video-conferencing decoder, comprising: aconnection to an upstream encoder; a packet loss detection mechanismconfigured to detect a lost packet and to transmit one of a packet lossnotification message and a lost packet identifier; and one of asustained packet loss detection mechanism connected to the packet lossdetection mechanism and the upstream encoder, the sustained packet lossdetection mechanism configured to receive the packet loss notificationmessage and to inform the encoder that the packet loss notificationmessage indicates a presence of one of a first and a secondpredetermined upstream packet loss scenario, and an error concealmentdevice connected to the packet loss detection mechanism and configuredto replace a lost motion vector with a motion vector from a previousframe.
 46. A video-conferencing system, comprising: an upstream encoderon an upstream network; a downstream decoder on a downstream network;and a video conferencing interface device connecting the upstream andthe downstream networks, including a decoder connected to the upstreamencoder; an encoder connecting the decoder to the downstream decoder; apacket loss detection mechanism connected to the decoder and configuredto detect a lost packet and to transmit one of a packet lossnotification message and a lost packet identifier; and one of asustained packet loss detection mechanism connected to the packet lossdetection mechanism and the upstream encoder, the sustained packet lossdetection mechanism configured to receive the packet loss notificationmessage and to inform the upstream encoder that the packet lossnotification message indicates a presence of one of a first and a secondpredetermined upstream packet loss scenario, and an error concealmentdevice connected to the decoder and the packet loss detection mechanism,the error concealment device configured to replace a lost motion vectorwith a motion vector from a previous frame.
 47. The video-conferencingsystem of claim 46, wherein the encoder is connected to a secondsustained packet loss notification device configured to identify apresence of a first and a second predetermined downstream packet lossscenario, the encoder further configured to insert an INTER or INTRAblock according to one of a first and a second method corresponding tothe presence of the first and the second predetermined downstream packetloss scenario.
 48. A video-conferencing system, comprising: an upstreamencoder on an upstream network; a downstream decoder on a downstreamnetwork; and a video conferencing interface device connecting theupstream and the downstream networks, including a packet loss detectionmechanism configured to detect a lost packet and to transmit a lostpacket identifier; and an error concealment device connected to thepacket loss detection mechanism, the error concealment device includinga decoder and an encoder, the error concealment device configured toreplace a lost motion vector with a replacement motion vector from aprevious frame.
 49. The video-conferencing interface system of claim 48,wherein the error concealment device is further configured to decompressincoming packets of compressed video data, detect a lost packet ofdecompressed video data, estimate a lost packet motion vector from acorresponding previous frame motion vector, compress the estimated lostpacket motion vector, reset a residual corresponding to the lost packetmotion vector to a predetermined value, compress the reset residual,combine the compressed estimated motion vector and compressed resetresidual to form an estimated packet, and output the estimated packet tothe downstream network.
 50. The video-conferencing system of claim 48,further comprising: a sustained packet loss detection mechanismconnected to the packet loss detection mechanism and to an upstreamencoder, the sustained packet loss detection mechanism configured toreceive a packet loss notification message from the packet lossdetection device and to inform the upstream encoder that the packet lossnotification message indicates a presence of one of a first and a secondpredetermined upstream packet loss scenario.
 51. The video-conferencingsystem of claim 48, wherein the error concealment device is furtherconfigured to decompress incoming packets of compressed video data,detect a lost packet of decompressed video data, estimate a lost packetmotion vector from a corresponding previous frame motion vector,compress the estimated lost packet motion vector, reset a residualcorresponding to the lost packet motion vector to a predetermined value,compress the reset residual, combine the compressed estimated motionvector and compressed reset residual to form an estimated packet, andoutput the estimated packet for to the downstream network.
 52. Avideo-conferencing system, comprising: means for collecting one ofpacket loss notifications and a lost packet identifier; and one of meansfor determining if the collected packet loss notifications correspond toone of a first and a second predetermined packet loss scenario, andmeans for replacing video data lost in a current frame using motionvector information from a previous frame in response to the lost packetidentifier.
 53. The video-conferencing method of claim 52, furthercomprising: means for inserting one of an INTRA and an INTER blockaccording to a first and a second method if the collected packet lossnotifications correspond to the first and the second predeterminedpacket loss scenario, respectively.
 54. A video-conferencing errorconcealment method, comprising: decompressing incoming packets ofcompressed video data; detecting a lost packet of decompressed videodata; estimating a lost packet motion vector from a correspondingprevious frame motion vector; compressing the estimated lost packetmotion vector; resetting a residual corresponding to the lost packetmotion vector to a predetermined value; compressing the reset residual;combining the compressed estimated motion vector and compressed resetresidual to form an estimated packet; and outputting the estimatedpacket.
 55. A video-conferencing error concealment device, comprising:means for decompressing incoming packets of compressed video data; meansfor detecting a lost packet of decompressed video data; means forestimating a lost packet motion vector from a corresponding previousframe motion vector; means for compressing the estimated lost packetmotion vector; means for resetting a residual corresponding to the lostpacket motion vector to a predetermined value; means for compressing thereset residual; means for combining the compressed estimated motionvector and compressed reset residual to form an estimated packet; andmeans for outputting the estimated packet.
 56. A computer programproduct configured to host and provide instructions corresponding toanyone of the method steps of claims 19-36.
 57. A computer programproduct configured to host and provide instructions corresponding toanyone of the method steps of claim 54.